Adaptive Policies for Markov Renewal Programs

Renewal Theorems for Markov Chains

Adaptive Control of Constrained Markov Chains : Criteria and Policies

We consider the constrained optimization of a nite-state, nite action Markov chain. In the adaptive problem, the transition probabilities are assumed to be unknown, and no prior distribution on their values is given. We consider constrained optimization problems in terms of several cost criteria which are asymptotic in nature. For these criteria we show that it is possible to achieve the same o...

A Diffusion Approximation for Markov Renewal Processes

For a Markov renewal process where the time parameter is discrete, we present a novel method for calculating the asymptotic variance. Our approach is based on the key renewal theorem and is applicable even when the state space of the Markov chain is countably infinite.

Algorithms for Learning Markov Field Policies

We present a new graph-based approach for incorporating domain knowledge in reinforcement learning applications. The domain knowledge is given as a weighted graph, or a kernel matrix, that loosely indicates which states should have similar optimal actions. We first introduce a bias into the policy search process by deriving a distribution on policies such that policies that disagree with the pr...

Time-Sharing Policies for Controlled Markov Chains

We propose a class of non-stationary policies called \policy time sharing" (p.t.s.), which possess several desirable properties for problems where the criteria are of the average-cost type; an optimal policy exists within this class, the computation of optimal policies is straightforward, and the implementation of this policy is easy. While in the nite state case stationary policies are also kn...